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SEC Tools: What are Surveys of Enacted Curriculum? 



The Council of Chief State School Officers (CCSSO) partnered with Andrew Porter and John 
Smithson of Wisconsin Center for Education Research (WCER) to develop an advanced, in- 
depth approach to collecting and reporting data on the “enacted curriculum'’ in K-12 math and 
science, i.e. the actual subject content and instructional practices experienced by students in 
classrooms. The set of tools are called the Surveys of Enacted Curriculum (SEC). We have 
conducted research and testing of enacted curriculum survey tools with schools and teachers. As 
a result, we are now able to offer the survey tools and a range of related data services to states 
and districts. The two organizations are collaborating with the Learning Point Associates and 
TERC Regional Alliance to disseminate the SEC tools and services to education systems and to 
assist schools, districts, and states in using these new tools for improving K-12 education. 

The Surveys of Enacted Curriculum are designed to provide reliable, comparable data that are 
collected at the classroom level with teachers and students. The Surveys are available for English 
Language Arts, Mathematics, and Science at the elementary, middle, and high school levels. The 
SEC data analysis and reporting tools are intended to assist teachers, administrators, and 
policymakers with planning for instructional improvement in several ways: 

1. Align curriculum with standards and system- wide assessments. 

2. Monitor indicators of instruction and relationship to student achievement. 

3. Analyze differences in instruction and content across schools and classes, and identify 

improvement strategies through school leadership teams. 

4. Evaluate effects of initiatives, such as professional development, in changing math and 

science practices. 

The SEC tools and services have been developed with assistance from many educators and 
researchers. The data collection instruments were field-tested in several hundred schools. The 
research and development process has been supported by states, the National Science 
Foundation, and the U.S. Department of Education. 
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Services Available 



Teacher and student surveys: The survey instruments are designed for self-report by 
teachers at elementary, middle or high school levels. Two major components of the teacher 
surveys are: a) Subject content of instruction, and b) Instructional activities and teacher 
preparation. Student survey items assist in validation, and are typically administered to a sub- 
sample of classes. A school system could choose sections of the teacher surveys for their specific 
needs. The complete survey requires average of 1.5 hrs. for a teacher to complete. An on-line, 
web-based survey system is available. 

Data collection, input, and analysis: Field tests and prior data collection show distinct 
advantages of survey administration on-site. Teachers and administrators should receive 
orientation beforehand on the intended uses of data for evaluation and school improvement. 
Paper survey forms can be used, with data input through a scanner. The web-based survey 
provides an alternative data entry, analysis and reporting mode. Data analyses focus on 
differences in curriculum content and instructional practices across classrooms, schools, and 
districts. Between teacher comparisons are possible. Item profiles and scales are analyzed by 
characteristics of schools (e.g., level, enrollment, reform initiative, student demographics) or by 
teacher characteristics (e.g., high professional development in subject vs. low professional 
development). 

Reporting formats useful to educators'. User-friendly graphics and charts have been 
developed to facilitate comparison of math and science instruction with state standards or local 
goals for improvement. School leadership teams, curriculum specialists, administrators, and 
policymakers can use the survey data reports to track effects of initiatives on classrooms, or 
identify areas of improvement efforts. The CCSSO/WCER analysis and report methods include 
analysis of alignment of instruction with state or local assessment instruments. 
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Grade 8 Mathematics 
Measurement Sub-Topics 
Alignment Analysis 
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Designs for professional development and assistance to schools'. The SEC collaborative 
provides models for workshops with teachers and school leaders based on use of enacted 
curriculum data and reports. Our designs include strategies for leadership in analysis of data 
within schools and methods of using cooperative inquiry-based approaches for improving 
instruction (see CCSSO website for links to presentations on several models: 
www.SECsurvey.org) . CCSSO is leading a three-year study, with support from the National 
Science Foundation, to test the effects of using enacted curriculum data with school teams as a 
strategy for professional development to improve instruction in math and science. The model is 
being tested with 40 middle schools in large, urban districts. Other models for using the 
curriculum data and integrating with existing professional development are available. 

• All of the SEC tools and services, and our list of publications, can be reviewed on the 

CCSSO SEC website — http://www.SECsurvey.org 

• An online, web-based version of the Surveys and reports is now available through 

Wisconsin Center for Education Research — http:///www.SEConline.org 

• Learning Point Associates established a website for increasing access of SEC for local 

educators http://www.SECsupport.org/ 

• The SEC mathematics and science instruments and materials are also available on 

compact disc — SEC- CD — to provide locally-directed surveys and applications (Contact 

CCSSO publications, 202/336-7016; www.CCSSO.org/Publications) 

Topics Covered by Surveys. The major concepts underlying the designs for the Surveys 
were drawn from state and national content standards, state initiatives in science and 
mathematics education, and prior research studies on classroom instructional practices and 
curriculum content. Each of the following topics can be addressed with the results of the Enacted 
Curriculum Surveys. The response data on each topic can be used to analyze instruction by the 
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amount of time spent teaching, differences among schools and classrooms, or by characteristics 
of students or differences in preparation of teachers. 



• Active Learning in Science 

• Problem Solving in Mathematics 

• Instructional Activities in classrooms, e.g., 
small groups, manipulatives, investigations 

• Mathematics and Science Content in 
Classrooms (see adjacent “content map” 
reporting format) 

• Multiple Assessment Strategies in Math 
and Science 

• Use of Education Technology and 
Equipment 

• Teacher Preparation in Subject 

• Quality of Professional Development 

• Influences of Policies and Standards on 
Practice 

• Alignment of Content and Standards / 
Assessments 




SAMPLE SECTIONS FROM SURVEY: MIDDLE SCHOOL MATHEMATICS 



Time on Middle School Mathematics 

Topic Topics 
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Research and Development 



Survey Methodology 

The curriculum surveys and data analysis services being offered were developed through 
CCSSO’s State Collaborative on Assessment and Student Standards (SCASS) on Surveys of 
Enacted Curriculum, and through research and development conducted by Porter and Smithson 
at the Wisconsin Center. A multi-state team of educators, assessment specialists, and 
researchers worked with project leaders at CCSSO and WCER to write, review, and field test 
math and science surveys over a four-year period (1994-1998). The development work for 
math and science was supported by states and the National Science Foundation. The 
development of the English language arts version of the surveys and reports was developed in 
2002-03 through support of states, U.S. Department of Education, and Learning Point 
Associates/ NCREL. 

The surveys are based on state and national standards for content and teaching, as well as prior 
well-tested survey instruments, including TIMSS, NAEP, and National Survey of Science and 
Math Education (Horizon Research). Survey instruments were thoroughly field tested to ensure 
reliability and validity of the data. 

A large field study was conducted with schools and teachers in 1 1 states to test and improve 
both the survey instruments and the data reporting methods and formats. More than 600 
teachers completed self-report surveys that covered the subject content they taught and the 
instructional practices they used in their classes. From 2001-04, a CCSSO-led team conducted 
an experimental design study with 40 urban middle schools to develop and test a professional 
development model for improving instruction in math and science through use of Data on 
Enacted Curriculum (Blank, 2004) 

Teachers participating in the Surveys report on a full school-year of teaching in English 
language arts, science or math. The data are analyzed and reported using pre-designed and 
tested charts and graphs found to be accessible and useful to educators as well as researchers. 
The survey method was validated by comparing teacher survey results, which require recall of 
instructional content and practices for a full school-year, with teacher reports using daily logs, 
independent observation, and teacher survey reports. This study found that data reported about 
curriculum content in teacher surveys covering a whole year were highly correlated with the 
data from daily logs of instructional content (Smithson and Porter, 1994). 

Findings from Research and Development 

Results from existing studies with Surveys of Enacted Curriculum show that educators can use 
the surveys and methods of analyzing data to address several kinds of problems and needs 
(Blank, et al., 2001; Porter, 2002; Blank, 2004). Specifically, the research studies show that 
SEC instruments and reports do address educators’ needs for comparable, reliable data and 
analyses of math and science instruction. The results show that the data: 
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• Demonstrated a high degree of variation in instructional time, practices and content of 
math and science from state to state, from school to school, and within schools. 

• Curriculum taught in math and science differed according to amount of standards- 
based professional development in schools and level of implementation of state reform 
initiatives. 

• The method of surveying teachers using a “content matrix” based on standards, and 
then analyzing content of assessments using the same tool, showed strong potential for 
application in districts and states for measuring progress of standards-based math and 
science. 

• The comprehensive design of the surveys proved useful in analyzing the relationship 
of curriculum content and pedagogy to level of preparation of teachers, professional 
development, and school conditions for teaching. 

(See, Blank, Porter, & Smithson, New Tools for Analyzing Teaching, Curriculum and 
Standards in Mathematics & Science, CCSSO, 2001). 

To view this report and other SEC reports and materials, go to the CCSSO website: 
http://www.ccsso.org/projects/Surveys of Enacted Curriculum. 
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Applications - Needs of Educators Addressed by SEC 



For education decision-makers, we group applications of SEC Tools and Services into five 
categories; no doubt other groupings are possible. We find that these five categories address 
the main types of needs and problems in K-12 education for which these tools and services are 
appropriate and helpful: 

1) Monitor indicators of instruction and relationship to student achievement 

Data are collected using standard, self-report teacher surveys across classrooms and schools to 
allow comparison of science and math instructional practices and subject content. Standard 
instruments and survey items are used. A web-based survey and data collection is now being 
tested, but paper forms are available. Data can be analyzed by characteristics of schools (e.g., 
level, enrollment, reform initiative) by grade level within a school or across schools, or by 
teacher characteristics (e.g., high professional development in subject vs. low professional 
development). 

Student achievement data from local or state assessments can be reported along with the 
curriculum and instructional data from SEC tools. Repeat administration and use of the surveys 
allows schools and districts to analyze extent of change or improvement in instructional 
practices, and achievement, over time (e.g., annual survey or every two years). Monitoring of 
change in practices can be school or by teacher (if teachers are identified). 

Components of SEC for this application are: 

• Planning (local leaders gain information about 
surveys and services, plan their use) 

• Orientation of Users (introduce surveys to 
administrators and teachers, demonstrate data 
uses) 

• Data Collection with Surveys (on-line or paper) 

• Analysis and Report on Instruction and 
Achievement Results (analyze teacher 
responses on practices and subject content; 
guide for data interpretation; graphics for data 
display) 

• Repeat data collection and reporting annually, 
or biennially. 



Using Data on Enacted Curriculum • 

A Guide tor Protessional Development 
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2) Obtain survey instruments to collect enacted curriculum data 

Districts or states may want to obtain the SEC survey instruments and collect and analyze the 
data using the data analysis methods and formats developed by CCSSO and WCER. The 
surveys are available on a SEC-compact disc (CD) as well as the analysis and report format, 
and they will be available on the CCSSO website. 

Options: 



• Purchase SEC- CD (from CCSSO Publications) 

• Access the CCSSOAVCER on-line SEC web-based surveys and report system 
http://www.SEConline.org 

Local districts or states would administer the surveys and conduct analysis of data using the 
models and data software provided on the CD or by gaining access to the on-line version of the 
tools. 

3) Align curriculum with standards or assessments 

Almost every state, and many districts, has developed content standards for student learning. 
The enacted curriculum surveys can provide a database for monitoring the degree to which 
classroom curriculum is moving toward the standards. States and districts are finding that one 
of the key benefits of the SEC tools is the capacity for analyzing the degree of intersection 
between the science and math subject curriculum content as taught in classrooms and content 
as tested on statewide or district-wide assessments. Procedures have been developed for 
content coding the assessments into the same content matrix used for the teacher survey 
providing teacher-reported data on what is taught. A mapping software from MS Excel is used 
to portray the degree of alignment between instructional content and assessed content. We 
recommend that districts organize a workshop for schools and teachers to analyze and interpret 
the results. 

Components: 

• Planning 

• Orientation of users 

• Data collection 

• Analysis and report, including Alignment analysis 

• Interpretation of data through Workshop with school leadership teams and teachers 

See Porter and Smithson’s recent paper (2002) for explanation of the SEC content coding and 
alignment methodology and procedures. 
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4) Analyze differences in instruction among schools and classrooms, and assist school 
leadership teams to identify instructional improvement strategies 

CCSSO, WCER, and TERC recently conducted a project with 40 middle schools aimed toward 
developing a model for improving instruction in math and science with use of the enacted 
curriculum data (Blank, 2004). The project was supported by a grant from National Science 
Foundation. 

The model involved school leader teams, teacher surveys, data reports for each school, school- 
based workshops, and technical assistance. The professional development leadership and 
technical assistance was provided by TERC Regional Alliance, and the model was based on 
their prior work on data-driven instructional improvement (Love, 2000). In year 2 and 3 of the 
study, the team collected data from teachers to measure the extent of change in instructional 
practices and curriculum content. 

Components: 

• Planning 

• Orientation of users 

• Data collection 

• Analysis / Report— S-M Instruction, Alignment 

• Interpretation of data / PD Workshop 

• Technical assistance (visits to schools over 18 month period, follow-up workshops) 

• Repeat data collection and analysis after 2 years 

5) Evaluate standards-based initiatives, e.g., professional development, to assess needs, 
determine effects, and assist planning 

Educators and decision-makers need reliable tools for evaluating the effects of initiatives 
aimed toward improving teaching practices and increasing student learning. The enacted 
curriculum surveys can provide a tool for identifying needs, determining effects on curriculum, 
or planning programs. In science and math, many states and districts have developed 
professional development models to improve teachers’ knowledge of subject content and their 
skills in teaching math and science. Methods of evaluating change in practices that provide 
valid, comparable results are difficult to design and implement. 

The SEC tools provide efficient survey-based methods of collecting data on current teaching 
practices, characteristics of teachers, and information on their professional development 
experiences, and then tracking change over time through repeat administrations of surveys. The 
effects of initiatives can be compared across schools and districts, and differing characteristics 
of teachers. 
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Components: 

• Planning 

• Orientation of users 

• Data collection 

• Analysis / Report— S-M Instruction and Experience of teachers with initiative 

• Interpretation of data in Workshop (survey reports provide key step in improving PD 
process) 

• Repeat data collection and analysis at end of initiative, or after one year or two years 
experience 
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Operational Plan- Assisting education systems to use SEC tools and services 



Educators or administrators may have obtained information about the SEC tools and services in 
a variety of ways. Requests from education systems to obtain SEC tools and services can be 
made through one of three routes: 

Direct contact inquiry for information or assistance to one of the staff at CCS SO or WCER 
by phone, email, or letter. In response, we would: 

• Discuss the requested services, and needs to be met, with users by phone or email 

• Email sample materials and descriptions of services 

• Negotiate arrangements to use surveys and services, determine method of support and 
payment, set schedule for providing services, send contract to user (see attached 
spreadsheet analysis of costs) 

• Project manager (from CCSSO or WCER) will be assigned to each participating site 

Purchase SEC-CD (containing all survey instruments, reports, papers, and professional 
development materials); or purchase hard-copy report or other publications from CCSSO. 

• Contact CCSSO publications office by phone or through CCSSO website 

Access On-line, Web-based SEC system - Steps in conducting data collection and receiving 
reports: 

• School users review surveys and report options; make decisions on grades, subject, 
sections to be used; and complete request form on-line (sent to central server/ data 
manager for processing) 

• Manager confirms requested services and arrangements, and transmits contract amount 

• Method of payment (e.g., purchase order or credit card) returned by user 

• Access code provided by manager to user for web surveys 

• Procedures and schedule relayed to user 

• Names and credentials of staff/consultants, and contact information are provided for 
partners that will provide assistance with local analysis/interpretation, technical 
assistance or workshop services 

• Teachers complete surveys on-line within designated time period 

• On-line feedback to users with comparison of individual responses with total sample 

• Data analysis based on pre-designed format conducted centrally, if requested and paid 

• Analytical report delivered by email attachment - based on school/district responses 
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Costs 



Estimates of the costs for obtaining SEC tools and services under each category are available 
from CCSSO. The cost estimates for each separate component are available as well. We 
encourage users to select the model or application that fits local or state needs and to arrange 
for the components needed in their system. See the CCSSO website for cost breakdown of the 
SEC tools and services- http://www.SECsurvey.org . 

For assistance in planning for your district or state, contact Rolf K. Blank, CCSSO Director of 
Education Indicators, rolfb@ccsso.org (202/336-7044), or one of the contacts with our partner 
organizations. 
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Key Contacts 



Rolf K. Blank (SEC Project Director) 

Council of Chief State School Officers 
One Massachusetts Ave. NW, Suite 700 
W ashington, DC 2000 1-1431 
202/336-7044; Fax 202/789-1792 
rolfb@ccsso.org 

Carlise Greenfield 

ccsso 

SEC Information Contact 

202/336-7066 

carliseg@ccsso.org 

John Smithson 

Wisconsin Center for Education Research 
1025 W. Johnson St. 

Madison, WI 53706 
608/263-44354; Fax 608/262-7428 
iohns@education.wisc.edu 

Jean Hess 

Learning Point Associates 
1129 E. Diehl Road 
Suite 200 

Naperville, IL 60563 
630-649-6623; Fax 630-649-6700 
Jean.hess@leamingpt.org 

Diana Nunnaley 

TERC Regional Alliance 
2067 Mass. Ave 
Cambridge, MA 02140 
617/873-9793 
Diana nunnaley@terc.org 
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